Endoscope with wide angle lens and adjustable view

ABSTRACT

A system comprises an endoscope with a wide-angle lens and a camera. An image of a field of view is captured by the endoscope and a portion of the image is displayed on the display. A popup image is generated and displayed upon image analysis identification of an occurrence of bleeding or motion of edges of an incision that is within the field of view, but outside the portion of the field of view displayed on the display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.15/232,836, filed Aug. 10, 2016, now U.S. Pat. No. 10,932,547, which isa Continuation-in-Part of International Application No.PCT/IL2015/050345, filed Mar. 31, 2015, claiming priority from U.S.Provisional Patent Application No. 61/973,906, filed Apr. 2, 2014, andContinuation-in-Part of U.S. Provisional Patent Application No.62/203,936, filed Aug. 12, 2015, the disclosures of which are herebyincorporated by reference in their entirety.

FIELD OF THE INVENTION

The present invention generally pertains to a system and method forproviding an endoscopic image, where the field of view of the image canbe maneuvered digitally.

BACKGROUND OF THE INVENTION

Laparoscopic surgery is becoming increasingly popular with patientsbecause the scars are smaller and their period of recovery is shorter.Laparoscopic surgery requires special training of the surgeon orgynecologist and the theatre nursing staff. The equipment is oftenexpensive and is not available in all hospitals. During laparoscopicsurgery it is often required to shift the spatial placement of theendoscope in order to present the surgeon with the optimal view.Conventional laparoscopic surgery makes use of either human assistantsthat manually shift the instrumentation or alternatively roboticautomated assistants (such as JP patent No. 06063003).

In laparoscopic surgery, the surgeon performs the operation throughsmall holes using long instruments and observing the internal anatomywith an endoscope camera. The surgeon's performance is largely dependenton the camera position relative to the instruments and on a stable imageshown at the monitor. In general, the surgeon needs a close-up view ofthe area in which he wants to work, however, there are times when anoverview of a large portion of the working area, such as an overall viewof the interior of the abdomen, is desirable. U.S. Pat. No. 6,387,044discloses a laparoscope apparatus for use in laparoscopic surgery or thelike comprises a cannula with an inner hollow having therein a lightguide for introducing light to its tip end for illuminating the objectto be observed, and an endoscope which is capable of being pulled intoand out from the inner hollow of the cannula. The cannula is airtightlysealed with a transparent member at its tip end, while the endoscope hastherein an image pickup with a wide-angle lens, and the image pick-up ishoused in the cannula close to the transparent member.

However, in U.S. Pat. No. 6,387,044, no means of adjusting magnificationof the image is disclosed.

Korean Patent Application KR2009/012326 discloses a laparoscope and animage processing system using the same to efficiently perform alaparoscope operation by outputting an image of the inside of theabdominal cavity. The laparoscope includes a wide angle lens, an opticalfiber, an inserting part, and an optical interface part. The wide anglelens is arranged at one end of the laparoscope, and is a fisheye lens.The focal distance of the wide angle lens is controllable. The opticalfiber delivers incident light from the wide angle lens to the camerapart, and is received in an inner space of the inserting part. Theinserting part is inserted inside the abdominal cavity. The opticalinterface part delivers the light delivered from the optical fiber tothe camera part. The camera part is a two dimensional camera or a threedimensional camera.

However, Korean Patent Application KR2009/012326 requires camera wideangle lens of adjustable focal length.

U.S. Patent Application US2011069160 discloses an imaging system andmethod of application, including lens designs tailored to be used withparticular transformation algorithms, electronic hardware and algorithmsfor image transformations. Exemplary application of the system includingautomotive, photographic and medical endoscopic are also described. Thesystem enables improved image view and allows customization of views bythe end user even after installation of the image system hardware. InU.S. Patent Application US2011069160, mathematical algorithms are usedto alter the image, so that a continuously varying distortion of theimage is possible, so that desired portions of the image are magnified,but the whole of the image remains within the field of view.

However, US Patent Application US2011069160 discloses a system whereinthe image is distorted, with the distortion changing continuously acrossthe image, so that portions of the image are magnified while theentirety of the field of view is displayed.

Chinese Utility Model CN203042209 discloses a laparoscopic puncturedevice with an intra-cavity full-view auxiliary lens, which relates to asurgical instrument device, in particular to the laparoscopic puncturedevice with the intra-cavity full-view auxiliary lens for abdominalcavity minimally-invasive surgery. A main lens of a laparoscope entersinto the abdominal cavity through a cannular puncture device, thepuncture device is provided with an auxiliary lens which can be used forobserving intra-cavity full view, and the auxiliary lens is a wide-anglelens and cold light source combination body; the auxiliary lens isarranged at the middle part of the puncture device entering into theabdominal cavity, and the auxiliary lens is positioned at the highestposition of the abdominal cavity after the abdominal cavity is subjectedto air inflation; the outer ring of the puncture device is provided witha groove, the wide-angle lens and cold light source combination body ofthe auxiliary lens is embedded in the groove and is combined into awhole with the puncture device; and the wide angle of a wide-angle lensof the auxiliary lens is larger than or equal to 270 degrees, and theresolution ratio of the wide-angle lens of the auxiliary lens is 1-3million pixels. By adding a cold light source and the wide-angle lens,the full-view visible function is added on the basis of the originalfunction of the puncture device, 80 percent of the abdominal cavity canbe within a visible range, motion of all instruments is within thevisible range, the surgical blind area of a surgeon can be removed, andthe surgery can be safer.

However, Chinese Utility Model CN203042209 requires use of at least twolenses.

It is therefore a long felt need to provide an endoscopic image, wherethe field of view of the image can be maneuvered digitally.

SUMMARY OF THE INVENTION

It is an object of the present invention to disclose a system forproviding an endoscopic image, where the image can be maneuvereddigitally.

It is another object of the present invention to disclose a system foraltering the field of view of an endoscope image, comprising:

-   a. at least one endoscope having at least one wide-angle lens in    said endoscope's distal end;-   b. at least one camera located in said endoscope's proximal end,    configured to image a field of view of said endoscope image by means    of said wide-angle lens; and-   c. at least one computer program which, when executed by data    processing apparatus, is configured to select at least a portion of    said field of view;

wherein said portion of said field of view is selectable withoutnecessarily physically maneuvering said endoscope or said wide-anglelens such that substantially all of said maneuvering of said field ofview is provided via said executing computer program.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said field of view can encompass asubstantial portion of a surgical field.

It is another object of the present invention to disclose the system asdescribed hereinabove, further comprising at least one display incommunication with said camera, said display configured to show at leasta portion of said selected portion of said field of view.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one computer program isfurther configured, when executed by data processing apparatus, toidentify from at least one said image of said field of view, theoccurrence of at least one predetermined item of interest.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said item of interest is selected from agroup consisting of: an article entering the field of view of the lens,an article moving, a likely collision between two articles, theoccurrence of bleeding, the edges of an incision moving, activation ordeactivation of a tool, articulation of a tool, and any combinationthereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein, if at least one said item of interest isidentifiable, a warning is activatable.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said warning is selected from a groupconsisting of a visual warning, an aural warning and any combinationthereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said aural warning is selected from agroup consisting of a predetermined voice message or a predeterminedsound.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said visual warning is selected from agroup consisting of a flashing light, a steady light, a region on saiddisplay changing in quality, and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said change in quality is selected from agroup consisting of: a changing color, a changing brightness, a pop-upappearing, an icon ungreying, a symbol ungreying, and any combinationthereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said display comprises at least onepredetermined location, said predetermined location configured todisplay, for a predetermined time, at least one said warning.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said item of interest is displayable as amember of a group consisting of: a display including said item ofinterest substantially entirely replaces a previous display, a displayfield of view which includes said item of interest appears as a popup, adisplay including said item of interest replaces a portion of a previousdisplay field of view and any combination thereof. It is another objectof the present invention to disclose the system as describedhereinabove, wherein digital zoom is selected from a group consistingof: straight enlargement, interpolating zoom and any combinationthereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein, for at least two camera pixels, forcases where at least a portion of each pixel maps onto at least aportion of a display pixel, said straight enlargement is selected from agroup consisting of averaging between said at least two camera pixels,interpolating between said at least two camera pixels and anycombination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein, in conjunction with said identificationof said occurrence of said at least one predetermined item of interest,a field of view encompassing said item of interest can be provided.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein at least a portion of a current field ofview is replaceable by said field of view encompassing said item ofinterest.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said replacement of said at least aportion of said current field of view is commandable via at least onepredetermined location on said display.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said camera is a high-resolution camera.

It is another object of the present invention to disclose the system asdescribed hereinabove, additionally comprising a database.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein at least one said portion of said fieldof view is storable in said database.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one computer program, whenexecuted, can alter the apparent size of said portion of said field ofview.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one computer program, whenexecuted, can show said portion of said field of view on said display.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said digital maneuvering is eithercontinuous or discrete.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said image of said at least a portion ofsaid field of view is substantially undistorted.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one computer program, whenexecuted by said data processing apparatus, is additionally configuredto perform said digital maneuvering automatically.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said automatic digital maneuvering isperformed in order to achieve a predetermined goal, said goal selectedfrom a group consisting of: maintaining a predetermined object at thecenter of said portion of said field of view, maintaining apredetermined object at a predetermined apparent size; maintaining apredetermined horizon, and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said system additionally comprises amaneuvering mechanism for physically maneuvering said endoscope.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one computer program, whenexecuted by said data processing apparatus, is additionally configuredto physically maneuver said endoscope by means of said maneuveringmechanism.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one computer program, whenexecuted by said data processing apparatus, is additionally configuredto maneuver said portion of said field of view by means of a groupconsisting of said physical maneuvering, said digital maneuvering andany combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said maneuvering of said field of view iscommanded by means of a member of a group consisting of: moving anobject, touching a prepared surface, typing on a keyboard, a gesture, anon-gesture body movement, generating a predetermined sound and anycombination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said movable object is selected from agroup consisting of: a joystick, a lever, a button, a slider and anycombination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said prepared surface is selected from agroup consisting of: a touch-sensitive pad displaying commands, adisplay comprising a touchscreen and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein selection of a center of said field ofview is by means of touching said touchscreen.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein selection of zoom is selected from agroup consisting of: drawing the outline of a desired image area on saiddisplay, continuing to touch said touchscreen until zoom is complete andany combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said gesture is selected from a groupconsisting of: a hand movement, an arm movement, a body movement, a headmovement, an eye movement and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said sound comprises at least onepredetermined sound pattern.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said predetermined sound pattern isselected from a group consisting of a word, a sound of constant pitch, asound of varying pitch, a sound of constant loudness, a sound of varyingloudness and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one movement of said atleast one moving element is an unwanted movement of said moving element.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said output procedure for movement ofsaid surgical tool, upon detection of said at least one said unwantedmovement of said moving element, is that, for said moving element beingsaid surgical tool, said unwanted movement is removed from said movementof said surgical tool; or, for said moving element not being saidsurgical tool, movement of said surgical tool is unaffected by saiddetection of said at least one unwanted movement.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said unwanted movement is selected from agroup consisting of: involuntary movement of a body part, saccadicmovement of an eye, vestibulo-ocular movement of an eye, winking an eye,blinking an eye, tremor of a body part, a tic in a body part, myoclonusof a body part, dystonia, and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said at least one camera generates aplurality of images.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said plurality of images is combinable toform a display selected from a group consisting of: a unitary twodimensional display; a unitary three dimensional display; a stereoscopicdisplay and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, additionally comprising at least one illuminationcontrolling computer program configured, when executed, to control atleast one aspect of illumination of said field of view.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said controllable aspect of saidillumination is selected from a group consisting of: number of sourcesof said illumination, intensity of a source of said illumination,wavelength range of a source of said illumination, number of light beamsfrom a source of said illumination, direction of at least one light beamfrom a source of said illumination, angular width of at least one lightbeam from a source of said illumination, diameter of at least one lightbeam from a source of said illumination and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein said wavelength range is selected from agroup consisting of: at least a portion of an infrared wavelength range,at least a portion of a visible wavelength range, at least a portion ofan ultraviolet wavelength range, at least a portion of an X-raywavelength range and any combination thereof.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein, for each said source of saidillumination, intensity of said illumination is selected from a groupconsisting of stepwise variable or substantially continuously variable.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein, for each said source of saidillumination, at least a portion of said source of said illumination isheld by at least one light holder.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein at least one said light holder is anendoscope.

It is another object of the present invention to disclose the system asdescribed hereinabove, wherein at least one said source of saidillumination is directable via a wide-angle lens.

It is another object of the present invention to disclose a method foraltering the field of view of an endoscope image, comprising steps of:

a. providing a system for altering the field of view of an endoscopeimage, comprising:

-   -   i. at least one endoscope having at least one wide-angle lens in        said endoscope's distal end;    -   ii. at least one camera located in said endoscope's proximal        end, configured to image a field of view of said endoscope image        by means of said wide-angle lens; and    -   iii. at least one computer program which, when executed by data        processing apparatus, is configured to select at least a portion        of said field of view;        b. imaging said field of view of said endoscope image; and        c. selecting said at least a portion of said field of view

thereby selecting said portion of said field of view without necessanlphysically maneuvering said endoscope or said wide-angle lens such thatsubstantially all of said maneuvering of said field of view is providedvia said executing computer program.

It is another object of the present invention to disclose the method asdescribed hereinabove additionally comprising step of encompassing asubstantial portion of a surgical field in said field of view.

It is another object of the present invention to disclose the method asdescribed hereinabove additionally comprising steps of providing atleast one display in communication with said camera, and of showing atleast a portion of said selected portion of said field of view on saiddisplay.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of configuring saidat least one computer program, when executed by data processingapparatus, to identify from at least one said image of said field ofview, the occurrence of at least one predetermined item of interest.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saiditem of interest from a group consisting of: an article entering thefield of view of the lens, an article moving, a likely collision betweentwo articles, the occurrence of bleeding, the edges of an incisionmoving, activation or deactivation of a tool, articulation of a tool,and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of activating awarning if at least one said item of interest is identifiable.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidwarning from a group consisting of a visual warning, an aural warningand any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidaural warning from a group consisting of a predetermined voice messageor a predetermined sound.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidvisual warning from a group consisting of a flashing light, a steadylight, a region on said display changing in quality, and any combinationthereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidchange in quality from a group consisting of: a changing color, achanging brightness, a pop-up appearing, an icon ungreying, a symbolungreying, and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising steps of providing saiddisplay with at least one predetermined location, and of displaying atsaid predetermined location, for a predetermined time, at least one saidwarning.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saiddisplay of said item of interest from a group consisting of: a displayincluding said item of interest substantially entirely replacing aprevious display, a display field of view which includes said item ofinterest appearing as a popup, a display including said item of interestreplacing a portion of a previous display field of view and anycombination thereof

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting digitalzoom from a group consisting of: straight enlargement, interpolatingzoom and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidstraight enlargement, for at least two camera pixels, for cases where atleast a portion of each pixel maps onto at least a portion of a displaypixel, from a group consisting of averaging between said at least twocamera pixels, interpolating between said at least two camera pixels andany combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of providing, inconjunction with said identification of said occurrence of said at leastone predetermined item of interest, a field of view encompassing saiditem of interest.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of replacing atleast a portion of a current field of view by said field of viewencompassing said item of interest.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of commanding saidreplacement of said at least a portion of said current field of view viaat least one predetermined location on said display.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidcamera to be a high-resolution camera.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of providing adatabase.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of storing at leastone portion of said field of view in said database.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of altering theapparent size of said portion of said field of view.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of showing saidportion of said field of view on said display.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of performing saiddigital maneuvering either continuously or discretely.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of providing saidimage of said at least a portion of said field of view substantiallyundistorted.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of configuring saidat least one computer program, when executed by said data processingapparatus, to perform said digital maneuvering automatically.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of performing saidautomatic digital maneuvering in order to achieve a predetermined goal,said goal selected from a group consisting of: maintaining apredetermined object at the center of said portion of said field ofview, maintaining a predetermined object at a predetermined apparentsize; maintaining a predetermined horizon, and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of providing saidsystem with a maneuvering mechanism for physically maneuvering saidendoscope.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of configuring saidat least one computer program, when executed by said data processingapparatus, to physically maneuver said endoscope by means of saidmaneuvering mechanism.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of configuring saidat least one computer program, when executed by said data processingapparatus, to maneuver said portion of said field of view by means of agroup consisting of said physical maneuvering, said digital maneuveringand any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of commanding saidmaneuvering of said field of view by means of a member of a groupconsisting of: moving an object, touching a prepared surface, typing ona keyboard, a gesture, a non-gesture body movement, generating apredetermined sound and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidmovable object from a group consisting of: a joystick, a lever, abutton, a slider and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidprepared surface from a group consisting of: a touch-sensitive paddisplaying commands, a display comprising a touchscreen and anycombination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting acenter of said field of view by means of touching said touchscreen.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidzoom by means of a member of a group consisting of: drawing an outlineof a desired image area on said display, continuing to touch saidtouchscreen until zoom is complete and any combination thereof.

The digital maneuvering of the FOV can also be performed automaticallyby an algorithm in order to achieve a predefined goal such as (but notlimited to) centering a certain object in the image, zooming in\out inorder to maintain a certain object size, rotating the image in order tomaintain a constant horizon.

The digital maneuvering of the FOV can be combined with physicalmaneuvering of the laparoscope\endoscope by means of controlling thelaparoscope position and orientation. The combination can be used inorder to achieve more flexibility in the viewed image, such as when thelaparoscope is inserted deeper in order to view beyond an occludedorgan, while the FOV is digitally maneuvered in order to tilt theviewing angle.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidgesture from a group consisting of: a hand movement, an arm movement, abody movement, a head movement, an eye movement and any combinationthereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidsound to be a predetermined sound pattern.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidpredetermined sound pattern from a group consisting of a word, a soundof constant pitch, a sound of varying pitch, a sound of constantloudness, a sound of varying loudness and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting said atleast one movement of said at least one moving element to be an unwantedmovement of said moving element.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidoutput procedure for movement of said surgical tool, upon detection ofsaid at least one said unwanted movement of said moving element, to bethat, for said moving element being said surgical tool, said unwantedmovement is removed from said movement of said surgical tool; or, forsaid moving element not being said surgical tool, movement of saidsurgical tool is unaffected by said detection of said at least oneunwanted movement.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidunwanted movement from a group consisting of: involuntary movement of abody part, saccadic movement of an eye, vestibulo-ocular movement of aneye, winking an eye, blinking an eye, tremor of a body part, a tic in abody part, myoclonus of a body part, dystonia, and any combinationthereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of generating aplurality of images from said at least one camera.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising steps of combining saidplurality of images to form a display and of selecting said display froma group consisting of: a unitary two dimensional display; a unitarythree dimensional display; a stereoscopic display and any combinationthereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising steps of providing atleast one illumination controlling computer program and of controllingat least one aspect of illumination of said field of view via saidillumination controlling computer program.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidcontrollable aspect of said illumination from a group consisting of:number of sources of said illumination, intensity of a source of saidillumination, wavelength range of a source of said illumination, numberof light beams from a source of said illumination, direction of at leastone light beam from a source of said illumination, angular width of atleast one light beam from a source of said illumination, diameter of atleast one light beam from a source of said illumination and anycombination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting saidwavelength range from a group consisting of: at least a portion of aninfrared wavelength range, at least a portion of a visible wavelengthrange, at least a portion of an ultraviolet wavelength range, at least aportion of an X-ray wavelength range and any combination thereof.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting, foreach said source of said illumination, intensity of said illuminationfrom a group consisting of stepwise variable or substantiallycontinuously variable.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of, for each saidsource of said illumination, holding at least a portion of said sourceof said illumination in at least one light holder.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of selecting atleast one said light holder to be an endoscope.

It is another object of the present invention to disclose the method asdescribed hereinabove, additionally comprising step of directing atleast one said source of said illumination via a wide-angle lens.

BRIEF DESCRIPTION OF THE FIGURES

In order to better understand the invention and its implementation inpractice, a plurality of embodiments will now be described, by way ofnon-limiting example only, with reference to the accompanying drawings,wherein

FIG. 1 schematically illustrates fields of view in the prior art;

FIG. 2 depicts a high-resolution camera image of an operating field inthe abdomen; FIGS. 3-5 depict close-ups which are enlargements ofportions of the image of FIG. 2 ;

FIG. 6A-G illustrates the effect of tip angle on the center of the fieldof view or the direction of motion of a physically zoomed endoscope forendoscopes with differently angled tips;

FIG. 7A-B illustrates a scene, both as it appears and as captured by acamera viewing it through a fisheye lens;

FIG. 8A-B illustrates a scene as captured by a camera and a field ofview as displayed; and

FIG. 9A-D illustrates a scene which includes an item of interest ascaptured by a camera, a display field of view showing a popup, a displayfield of view showing a warning and a display field of view whichencompasses the item of interest.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description is provided, alongside all chapters of thepresent invention, so as to enable any person skilled in the art to makeuse of said invention and sets forth the best modes contemplated by theinventor of carrying out this invention. Various modifications, however,will remain apparent to those skilled in the art, since the genericprinciples of the present invention have been defined specifically toprovide a means and method for providing a laparoscopic or endoscopicimage, which is digitally maneuverable.

The term ‘camera’ hereinafter refers to an image acquiring element.Examples of a camera include, but are not limited to, a CCD array and anelectromagnetic system such as a TV camera.

The term ‘endoscope tip’ hereinafter refers to the end of the endoscopethat is inside the patient. The camera is typically attached to theother side of the endoscope, outside of the patient's abdomen.

The term ‘field of view’ (FOV) hereinafter refers to the scene visibleto the camera. The term ‘displayed view’ hereinafter refers to the scenevisible on a display.

The term ‘digital maneuvering’ hereinafter refers to maneuvering thefield of view of an endoscope or laparoscope by using software tomanipulate the image, such that both the center of the field of view andthe extent of the field of view can be changed with or without physicalmovement of any portion of the endoscope or laparoscope, withoutphysical movement of any portion of the camera and without movement ofany lenses in the system.

Said digital maneuvering is provided by means of an image processingmeans that processes the image to provide change in the field of view.For example, if the image taken by the camera is an image of 270 degreesthe processor can provide the user with an image of 30 degrees out ofthe 270 degrees. The user can then alter the field of view to another 30degrees segment of said 270 degrees. All of this is done by imageprocessing and not by physically maneuvering the endoscope.

The terms ‘physical maneuvering’ or ‘optical maneuvering’ hereinafterrefer to maneuvering the field of view of an endoscope or laparoscope byphysically moving at least one of (a) some part of the endoscope orlaparoscope, (b) some portion of the camera, or (c) one or more lensesin the system.

The term ‘wide-angle lens’ hereinafter refers to any lens having a fieldof view of at least 30 degrees, preferably, at least 60 degrees, and, byextension, any endoscope containing such a lens. An example of anendoscope having a wide-angle lens is an endoscope capable of providinga 270 degree image.

Laparoscopic surgery, also called minimally invasive surgery (MIS), is amodern surgical technique in which operations in the abdomen areperformed through small incisions (usually 0.5-1.5 cm) as compared tolarger incisions needed in traditional surgical procedures. The keyelement in laparoscopic surgery is the use of an endoscope, which is adevice configured for viewing the scene within the body, at the distalend of the endoscope. The at least one camera can be placed at the endof the endoscope, a rod lens system or fiber optic bundle can be used todirect the at least one image to the proximal end of the endoscope, andany combination thereof. Typically, also attached to the endoscope is atleast one light source to illuminate the operative field, insertedthrough a 5 mm or 10 mm cannula or trocar to view the operative field.

The abdomen is usually injected with carbon dioxide gas to create aworking and viewing space. The abdomen is essentially blown up like aballoon (insufflated), elevating the abdominal wall above the internalorgans like a dome. Within this space, various medical procedures can becarried out.

For simplicity, in the embodiments hereinbelow, the system is describedas producing a single image from a single camera. However, any of theembodiments described herein can comprise one or a plurality of camerasand, for any camera, one or a plurality of images can be produced.

In embodiments with a plurality of cameras, the images therefrom can becombined to form a unitary two dimensional display; the images therefromcan be combined to form a unitary three dimensional display; the imagestherefrom can be used to provide a stereoscopic display, simulating athree-dimensional display; and any combination thereof.

In some embodiments, at least one light source is controllable by thesystem. In preferred variants of these embodiments, the controllablefeatures of the light include, but are not limited to, the intensity ofthe light source, the wavelength range of the light source, thedirection of the beam for the light source, the angular width of thebeam, the diameter of the beam and any combination thereof. Wavelengthranges can include, but are not limited to, at least a portion of aninfrared wavelength range, at least a portion of the visible wavelengthrange, at least a portion of a UV wavelength range, at least a portionof an X-ray wavelength range and any combination thereof. For eachsource, light intensity can be stepwise variable (an on/off source hastwo steps) or substantially continuously variable (often described asinfinitely variable). Light can enter the surgical field via theendoscope, or from a separate illuminator.

In preferred variants of embodiments with a plurality of cameras, all ofthe cameras are comprised in a single endoscope, thereby preventingunexpected differential movement between cameras due to mechanical driftor movement error and ensuring that there is proper registration of theimages at all times.

In preferred variants of embodiments with a plurality of light sources,all of the light sources are comprised in a light source holder, therebypreventing unexpected differential movement between light sources due tomechanical drift or movement error and ensuring that the surgical fieldis properly illuminated at all times. In some embodiments, the endoscopecomprises the light source holder. In some embodiments, the endoscopecomprises both the light sources and the cameras, thereby obviating apossible source of error in illuminating regions viewed by at least onecamera.

In some embodiments, at least one light source is directed into thesurgical field via a wide-angle lens such that a significant portion ofthe surgical field, preferably substantially all of it, can beilluminated via the single wide-angle lens.

In some variants of embodiments where the light enters the surgicalfield via a wide-angle lens, at least one light source passes throughonly a portion of the lens. In such embodiments, the region of thesurgical field illuminated by the light source will depend on theportion of the lens through which the light passes; for a significantportion of the surgical field, preferably substantially all of it, abeam of light can be positioned in a desired region of the surgicalfield without need for moving the endoscope. In some variants ofembodiments with a light beam smaller than the lens, the light beam canbe moved relative to the lens and the lens can be moved relative to theendoscope, such that the angle at which the light enters the surgicalfield can be varied without need for movement of the endoscope.

In many cases, the endoscope cannot view the entire working space withinthe body, so the endoscope must be repositioned to allow the surgeon toview regions of interest within the space. However, moving the endoscopecarries with it the danger of the endoscope contacting a tool andpossibly moving it or, worse, contacting a portion of the body andpossibly damaging it.

In many cases, the surgeon wants a close-up view of the working area; inother cases, an overview is desirable and a rapid transition fromclose-up to overview and vice-versa is also desirable.

The device disclosed herein uses an endoscopic camera in conjunctionwith a wide-angle lens and software for viewing control in order toprovide an endoscopic system with digital maneuvering of the field ofview, wherein both the center of the field of view (the position) andthe extent of the field of view (the zoom) can be altered rapidly, inmany cases, without need for physical movement of any part of thesystem. Furthermore, digital maneuvering can be either continuous ordiscrete.

The advantages of digital maneuvering (digital zoom and digitalpositioning) include:

-   -   There is very little or no need to physically alter the position        of the endoscope in order to change the viewing angle, which can        be especially important in robotic systems where robotic control        of the viewing angle is used rather than human control.    -   Maneuvering of the endoscope is simplified—no moving parts are        needed either for control of endoscope position or for control        of components within the optical system.    -   Change in the viewing direction can be continuous, and discrete        steps are not of fixed size, unlike the fixed-size discrete        steps found in systems such as the Storz-EndoCameleon™.

In the Storz-EndoCameleon, manipulation of the field of view is carriedout mechanically. FIG. 1 illustrates the discrete, although overlapping,fields of view (110) possible with the Storz-EndoCameleon of the priorart.

In the device disclosed herein, the use of a wide-angle lens allowsprimarily digital maneuvering of the endoscope. A wide-angle lens suchas a fish-eye lens can provide an image of a large portion, if not all,of a working area, such as the interior of the abdomen. The imageprovided by a wide-angle lens is typically distorted; software can beused to correct the distortion.

A high resolution camera, preferably at least 4096×3072 pixels, canprovide sufficient detail for digital maneuvering; 4D maneuveringincluding zoom can be implemented in software, minimizing the need tophysically move the endoscope.

Two types of digital zoom are common in the art. In one, sometimescalled “intelligent zoom” or “iZoom” and referred to herein as“interpolating zoom”, for at least part of the zoom range, the camerahas higher resolution (more pixels per unit area of the object(s)viewed) than the display, so that, as the inward digital zoom progressesand the display image enlarges, the display shows more resolution, asdetail captured by the camera but not previously displayable becomesdisplayable.

The other common type of digital zoom will be referred to herein as“straight enlargement”. In straight enlargement, for at least part ofthe zoom range, the camera has a resolution the same as or smaller thanthe resolution of the display, so that one camera image pixel will bemapped to one or more display pixels. It should be noted that “more thanone display pixel” includes fractional numbers of pixels, where thefraction is greater than one. Straight enlargement can be carried out byany means known in the art, including, but not limited to, spreading thecamera pixel over all the display pixels; averaging two or more camerapixels in cases where some fraction of each pixels maps, in part, tosome fraction of a display pixel (none of the fractions need be thesame), interpolating between camera pixels and any combination thereof.

In typical embodiments described herein, a member of a group consistingof interpolating zoom, straight enlargement and any combination thereofwill be employed. Preferably, interpolating zoom will be employed.

A typical image taken with a high-resolution camera is shown in FIG. 2 ;the image shows the detail provided by large number of pixels in theimage from such a camera. It is the large number of pixels that enableszooming, since the image will remain clear and detailed even when asmall portion of it is zoomed to fill the whole display. In FIG. 2 ,locations are shown for the centers of the enlarged images of FIG. 3(210), FIG. 4 (220) and FIG. 5 (230).

FIG. 3 shows (210) an enlarged view of a portion of the image of FIG. 2, centered halfway between the tools. The enlarged image includesapproximately 10% of the area of the original picture but details, suchas the tools (215), remain clear. Again, said image is provided merelyby image processing, not by physically maneuvering (or zooming) theendoscope.

FIG. 4 shows (220) an enlarged view of a portion of the image of FIG. 2, centered on the lobe at the top right of the image of FIG. 2 . Theenlarged image includes approximately 10% of the area of the originalpicture but details, such as the blood vessels (225), remain clear.Again, said image is provided merely by image processing, not byphysically maneuvering (or zooming) the endoscope.

FIG. 5 shows (230) an enlarged view of a portion of the image of FIG. 2, centered on the lobe at the top right of the image of FIG. 2 . Theenlarged image includes approximately 10% of the area of the originalpicture, but details such as the blood vessels (235) remain clear.Again, said image is provided merely by image processing, not byphysically maneuvering (or zooming) the endoscope.

In the device of the present invention, software is used to correct anydistortion of the image caused by the lensing system, to digitally movethe image to a display a selected portion of the field of view (positionthe image) and to digitally alter the size of the display view, theviewed portion of the field of view (zoom the image).

In many endoscopic systems, the tip lens in the camera optics is not ata right angle to the sides of the endoscope. Conventionally, the tiplens angle is described relative to a right angle, so that a tip lens atright angles to the sides of the endoscope is described as having anangle of 0. Typically, angled endoscope tip lenses have an angle of 30°or 45°. This tip lens angle affects the image seen during physicalzooming. FIG. 6A-E illustrates, in an out-of-scale manner, for aconventional system, the effect of physical zooming in the field of viewin an endoscope with tip lens set straight in the end (FIGS. 6A and 6C)vs. the effect of physical zooming in the field of view in an endoscopewith angled tip lens (FIGS. 6D and 6E).

FIGS. 6A and 6E illustrate the endoscope (100), the object it is viewing(200) and the image seen by the endoscope camera (130) before thephysical zoom. The solid arrows (160) show the limits of the field ofview (FOV) and the dashed arrow (170), the center of the FOV; since theobject is in the center of the FOV, an image of the object (210) is inthe center of the camera image (130).

FIGS. 6B and 6E illustrate the endoscope (100), the object it is viewing(200) and the image seen by the endoscope camera (130) after thephysical zoom. The solid arrows (160) show the limits of the FOV and thedashed arrow (170), the center of the field of view.

If the tip lens is set straight in the end of the endoscope (FIGS. 6Aand 6B), an object (200) in the center FOV will be in the center of theFOV (and the camera image) (130) both before (FIG. 6A) and after (FIG.6B) the physical zoom. As illustrated in an out-of-scale manner in FIG.6C, the direction of motion of the endoscope during the physical zoom(180) is a straight line connecting the location of the center of thetip of the endoscope (100) at the start of the physical zoom with thecenter of the field of view at the start (and end) (170) of the physicalzoom; the center of the endoscope tip will lie on this line at all timesduring the physical zoom.

However, if the tip lens is set at an angle in the end of the endoscope(FIGS. 6D and 6E), then an object that is in the center of the FOV (andthe camera image) before the physical zoom (FIG. 6D) will not be in thecenter of the FOV (or the camera image) after the physical zoom (FIG.6E) since the direction of motion of the endoscope is not the directionin which the center of the field of view (170) points.

In some embodiments of the system of the present invention, thecontrolling means maintains a fixed center of the field of view (FOV)during physical zoom independent of the tip lens angle. In such systems,(FIGS. 6F and 6G) the endoscope (100) tip will move in a curvedtrajectory (180, FIG. 6F) such that the center of the FOV does notchange during physical zooming (FIG. 6G).

Maintenance of a fixed center of the field of view can be can be madeeither by inputting the angle of the tip lens during setup, in whichcase, the system can calculate an appropriate trajectory, or byidentifying the center of the field of view by analyzing the image.Inputting the tip lens angle means that no image analysis need be done;however, controlling the direction of motion of the endoscope duringphysical zoom via image analysis means that that the tip lens angle doesnot need to be input, obviating a possible source of error.

In preferred embodiments of the device of the present invention, bothdigital maneuvering and physical maneuvering are enabled so thatmaneuvering from one display view to another display view can includedigital maneuvering, physical maneuvering, or both physical and digitalmaneuvering.

In preferred embodiments, the physical and digital movements comprisingthe maneuvering are under software control. A surgeon or other userdirects the system to maneuver the display view; the software thencontrols the physical and digital movements needed to accomplish thedesired maneuver. Physical maneuvering of the laparoscope or endoscopeis accomplished by controlling the laparoscope's position andorientation. Combined digital and physical maneuvering can be used toachieve more flexibility in the viewed image, such as, for non-limitingexample, inserted the laparoscope deeper into the operating region inorder to view beyond an occluded organ, while digitally maneuveringdisplay view in order to tilt the viewing angle.

Digital maneuvering of the display view can also be performedautomatically by an algorithm in order to achieve a predefined goal suchas (but not limited to) centering a certain object in the image, zoomingin or out in order to maintain a certain object size, rotating the imagein order to maintain a constant horizon and any combination thereof. Fornon-limiting example, a surgeon can instruct the system to retain aspecified tool in the center of the display image. If the surgeon movesthe tool towards himself (and away from the operating site) and to theright, the display view will automatically zoom outward (shrinking theapparent size of the operating site) and will track to the right (movingthe operating site to the left in the display view).

The surgeon can control maneuvering in an accustomed manner, by moving amoving element, whether the movement of the moving element be by movingan object, by touching a prepared surface, by typing on a keyboard, by agesture, by a non-gesture body movement, by a sound signal, by anelectrical or magnetic signal indicating movement of at least onemuscle, by an encephalographic pattern indicating movement of at leastone muscle, by an encephalographic pattern indicating future movement ofat least one muscle, and any combination thereof.

For non-limiting example, the moved object can be a joystick, a lever, abutton or a slider.

For non-limiting example, the prepared surface can be a touch-sensitivepad with commands on it, or the display can comprise a touchscreen andthe surgeon can touch the location in the image which will form thecenter of the field of view, and can zoom by drawing the outline of thedesired image area on the display, or by holding his finger on thescreen until zoom is completed. Any combination of the above can also beused.

For non-limiting example, a gesture can comprise a hand movement, an armmovement, a body movement, a head movement, an eye movement, and anycombination thereof.

For non-limiting example, the sound can comprise a word, a predeterminedsound pattern such as a sound of predetermined pitch, either constant orvarying, a sound of constant loudness, a sound of varying loudness andany combination thereof.

In some embodiments, control of the tools and of maneuvering of thelaparoscope does not require physical contact between the surgeon andeither the tools or the laparoscope. Control of at least one tool and/ormaneuvering of an endoscope can be via at least one predetermined inputcommand associated with a predetermined output command or predeterminedoutput protocol, or via detection, by the system, of an input procedure,with the output procedure dependent on at least one of the inputprocedure and other information accessible to the system.

The system of the present invention can be used in a conventionalmanner, with the operator and other members of the operating team in thesame room as the patient during the operation, or the system of thepresent invention can be used for remote surgery, with the operatorcontrolling the laparoscope and the tools from a location remote fromthe patient. In addition, control of maneuvering of a tool or thelaparoscope can be done without a joystick or other object whichrequires the operator, during the operation, to place his hand incontact with the device.

In some embodiments, the system provides an override facility such thatan undesired movement can be overridden. The override can be a voicecommand, a movement, an intended movement or a thought. The movement orintended movement can be movement of a hand, an eye, an arm, a finger, achest, a neck, a head, a mouth, a tongue, vocal cords (a predeterminedsound), a leg, a toe, a foot or any combination thereof. An actualmovement can be detected by any movement detection means, as describedhereinbelow. An intended movement can be detected by means of muscularelectric or magnetic patterns, as described hereinbelow, or fromencephalographic patterns (“brain waves”), as described hereinbelow.Similarly an override thought can be detected by means ofencephalographic patterns.

In some embodiments, the system can identify at least one unwantedmovement protocol for at least one moving element. Non-limiting examplesof unwanted movement protocols include: involuntary movement of a bodypart, saccadic movement of an eye, vestibulo-ocular movement of an eye,winking an eye, blinking an eye, tremor of a body part, a tic in a bodypart, myoclonus of a body part, dystonia, and any combination thereof.

In such embodiments, the preferred response is for the system to ignorethe unwanted movement, so that the actual output movement is unaffectedby and substantially independent of the unwanted movement. Fornon-limiting example, in a system where the movement of an endoscope isproportional to movement of an eye, the jerkiness of the actual eyemovement, imparted both by saccadic movement and vestibule-ocularmovement, will be “programmed out” so that the movement of the endoscopeis smooth. Similarly, if eye movement is controlling movement of anendoscope to the right, a quick glance upward will be “programmed out”;the endoscope will not diverge from the direct path to the right.

In another non-limiting example, movement of two retractors iscontrolled by movement of two arms. During a retraction to further openan incision, the operator suffers a muscular twitch that jerks an armupward. The jerk is ignored so that the retractors move apart smoothly.

For non-limiting example, if the surgeon commands a movement of thecenter of the field of view in a given direction, the surgeon will seethe center of the field of view moving in that direction, althoughneither the endoscope nor the camera nor the lenses nor any of thecamera optics will have physically moved. Similarly, a command to zoomin on the center of the field of view will cause the image to zoom,enlarging the image of the center of the field of view while reducingthe portion of the image which is shown. Again, there will have been nophysical movement of the endoscope, of any lens, of the camera, or ofany component of the camera optics.

In preferred embodiments, discrete alteration of the field of view isalso enabled. Again, although there is digital movement—the image on thedisplay changes—no physical movement is involved; there was no physicalmovement of the endoscope, the camera, the lenses or any part of thecamera optics.

In embodiments with discrete alteration of the field of view, fornon-limiting example, a user may wish to have an overview of thesituation, by switching from a close-up of the area on which he isworking to view of a large portion of the interior of the abdomen. Thiscan be desirable, for non-limiting example, if the surgeon suspects thatthere may be bleeding and wishes to find the source of the bleeding inorder to stop it.

In some variants of embodiments with discrete alteration of the field ofview, switching is from (and back to) the current position and zoom,with switching to (and from) a single, predetermined, overview position,with a single, predetermined zoom and position. Preferably, this single,predetermined zoom and position includes in the image digitally all of aworking area such as, but not limited to, the interior of the abdomen.

In other variants of embodiments with discrete alteration of the fieldof view, the surgeon can select desired overview positions and zooms,selected overview positions and zooms being stored in a database. As anon-limiting example, at the beginning of an operation, the surgeon canexamine the image and create overviews by selecting desired positions,and, for each desired position, adjusting the zoom, then storing theresulting position and zoom in the database. During the operation, whena desired overview is selected, the image “jumps” so as to display animage with that overview's position and zoom. The surgeon can theneither jump back to the previous close-up view, or jump to anotheroverview.

In some embodiment, the system comprises predetermined locations on thedisplay, such that, by positioning a movable article such as, but notlimited to, a tool within the predetermined location, the systemperforms a predetermined action. The predetermined location can be thelocation on the display of an article, or it can be an icon or popup.Icons are preferably at the edges of the display, although they can beanywhere on the display or even in a separate display; popups can occuranywhere, including at the edge of the display, elsewhere on thedisplay, or on a separate display.

For non-limiting examples:

-   -   In some embodiments, at least one full-screen location is        provided, which commands a switch between any view and an        overview encompassing the entire field of view of the image.    -   In some embodiments, at least one “return to previous view”        location is provided, which commands a switch back to at least        one previous zoom and center.    -   In some embodiments, a single location switches between        full-screen mode and return-to-previous mode.    -   In some embodiments, the return-to-previous location can return        to a plurality of previous views. For non-limiting example, each        time the movable article enters the return-to-previous location,        the view switches back another view.    -   In some embodiments, there is at least one go-to-next location,        which commands a switch forward to at least one subsequent zoom        and center.    -   In some embodiments, there is at least one switch-between        location, which enables switching, as described above, between        predetermined overview positions and zooms.    -   In some embodiments, there is at least one warning location,        such that, as described hereinbelow, an operator can be warned        of an item of interest occurring outside of the displayed        region.    -   In some embodiments, there is at least one go-to location, such        that, as described hereinbelow, the region where the item of        interest is occurring can be displayed, either by enlarging the        field of view or by switching the center of the field of view to        the region where the item of interest is occurring.

In most embodiments of the system, the region viewed by the lens issignificantly larger than the region displayed. FIG. 7A shows anon-limiting example of a region (700) viewable by a fisheye lens, inthis case a railway station, while FIG. 7B shows the whole of the imageas captured by a camera using the fisheye lens (710).

FIG. 8A shows, on the fisheye lens image (710) the limits of the fieldof view of a display (720), while FIG. 8B shows the image as seen in thedisplay (720), with the distortions of the fisheye lens removed bysoftware.

In some embodiments of the system, image analysis is done so that thesystem “knows” what is happening outside the displayed image, but withinthe field of view of the lens. In such embodiments, the operator can beprovided with a warning if something of interest has occurred outsidethe field of view of the display, but within the field of view of thelens.

Items of interest can include, but are not limited to, an articleentering the field of view of the lens, an article moving, a likelycollision between two articles, the occurrence of bleeding, the edges ofan incision moving, activation or deactivation of a tool, articulationof a tool, and any combination thereof.

Non-limiting examples of collisions between two articles are: acollision between two or more tools, and a collision between at leastone tool and an organ.

The warning can be visual or aural, with an aural warning selected froma group consisting of a predetermined voice message or a predeterminedsound. A visual warning can be selected from a group consisting of alight, either flashing or steady, or a region on the display changing inquality, where the change in quality can include, but is not limited to,changing color, changing brightness, a pop-up appearing, an icon orother symbol ungreying, and any combination thereof.

FIG. 9A shows the scene of FIG. 7A, with the addition of an exemplaryitem of interest, a sword. FIG. 9B shows visual warning, a popup (950),indicating the existence of the item of interest (730) outside the fieldof view of the display (720). In this embodiment, the location of thepopup indicates that the item of interest is in the lower left quadrantof the field of view of the fisheye lens (710). A popup can be in afixed position, it can use an arrow or other directional symbol toindicate the direction of the item of interest with respect to the iconor with respect to a fixed position (such as the center of the field ofview), it can use different weight or different color symbols toindicate a distance to the item of interest, or a text messageindicating direction, distance or both. The text message can be on thewarning, or it can form part of a separate warning, which can be anytype of visual or aural message as described hereinabove for a warning.Any combination of the above warnings and/or direction indicators and/ordistance indicators can be used.

The system can provide preferred responses by means of which an operatorcan respond quickly to the warning. Such preferred responses caninclude, but are not limited to, moving the center of the field of viewto the region where the item of interest is occurring, with or withoutzooming to improve the view of the region; zooming outward so that thefield of view includes both the original field of view and the regionwhere the item of interest is occurring, and any combination thereof.

A preferred response can be selected by positioning a movable article,preferably a tool, in a predetermined region of the display, such as onan icon or popup; by a predetermined movement of a movable article,preferably a tool; by touching a predetermined location on a screen,such as an icon or popup and any combination thereof.

FIG. 9C shows a field of view of the display (725) which includes theitem of interest, as it would appear in some embodiments of the system.In this example, the magnification is the same for the original field ofview and the field of view including the sword. In other embodiments,other magnifications can be used.

In the embodiment shown, the field of view of the display (725) showingthe item of interest entirely replaces the previous field of view of thedisplay (720). In other embodiments, a display field of view whichincludes the item of interest can appear as a popup or can replace onlya portion of the earlier display field of view as in, for non limitingexample, a split screen.

It should also be noted how distorted the camera image is at thelocation of the item of interest, the sword, since the sword is near theedge of the field of view of the fisheye lens and of the camera.

It should be noted that software manipulation of the size and locationof the display field of view, and software manipulation of the size andlocation of the illuminated region minimize the need for hardwaremanipulation, thereby minimizing the size and complexity of manipulationhardware and maximizing the space within the endoscope and/or thesurgical field which is available for other purposes.

It should further be noted that in embodiments with automatic control ofthe size and location of the display field of view, no human assistantis needed to manipulate the field of view, thereby eliminating a sourceof human error.

It should also be noted that in embodiments with automatic control ofthe size and location of the illuminated region, no human assistant isneeded to manipulate the field of view, thereby eliminating a source ofhuman error.

The invention claimed is:
 1. A system for altering the field of view of an endoscope image, comprising: at least one endoscope comprising at least one wide-angle lens and a camera configured to image a field of view of said endoscope image by means of said wide-angle lens; a display; executable software configured to select at least a portion of said field of view without physically maneuvering said endoscope or said wide-angle lens and to display said portion of the field of view on the display; wherein said executable software is configured, when activated, to provide a popup image upon image analysis identification of an occurrence of bleeding or motion of edges of an incision within the field of view but outside the portion of said field of view, said popup image comprising at least one overlay on the portion of the field of view displayed on the display, wherein said overlay comprises an image of an area of said occurrence of bleeding or said motion and is positioned to indicate a location of the bleeding or motion relative to the portion of the field of view.
 2. The system of claim 1, wherein the system additionally comprises illumination controlling executable software configured, when executed, to control at least one aspect of illumination of said field of view when displayed on the display.
 3. The system according to claim 1, wherein said executable software is configured to automatically digitally move said image to display a selected at least a portion of said field of view to maintain a predetermined object at the center of said portion of said field of view.
 4. The system according to claim 1, wherein said executable software is configured to automatically digitally move said image to display a selected at least a portion of said field of view to maintain a predetermined horizon.
 5. The system according to claim 1, wherein said executable software is configured to automatically digitally move said image to display a selected at least a portion of said field of view to maintain a predetermined object at a predetermined apparent size.
 6. The system of claim 1, wherein said executable software is configured to automatically select a portion of a field of view; remove distortion from said field of view, and display said portion of said field of view on the display.
 7. The system of claim 1, wherein said camera generates a plurality of images.
 8. The system of claim 7, wherein said executable software is additionally configured to combine said plurality of images to form a display image, said display image selected from a group consisting of: a unitary two-dimensional display; a unitary three-dimensional display; a stereoscopic display and any combination thereof. 